Alright, is it it's ready. Okay, great. Yeah, everything's okay. Okay. Okay, okay, yeah. Okay, thanks. Okay. Okay, thanks. Yeah. Yeah, yeah a very or a quick uh slides. So um I've been here for fourteen month, I think. Yeah, so I've a accustomed to the environment here and enjoy staying here. So mm uh from uh last year I was uh uh s uh supervising the annotation of segmentation, summarisation uh uh with Jean uh of the I uh IC ICSI corpus and uh uh uh on the other hand uh I w n uh did s some very preliminary work on chunking the switchboard corpus. But uh that was not uh well picked up in the AMI corpus, which is supposed to be. So uh and currently I'm more interested in see uh uh, yeah, analysing the discourse structure in multi-party meetings. This is what I'm going to present here. So uh uh in this presentation I'm going to see how we could analyse utterance to utterance relations in multi-party meetings. Um 'Cause meetings is a lit different from normal text or a n a normal dialogue or monologue. So interactivity is a concept is a concept that I'm using to distinguish the two uh uh extremes of uh of interac uh of discourse in meetings. And uh for dialogues uh and monologues I think we might have to use different uh concepts to m m uh model the discourse structure. Then uh I w I will uh give a very uh rough plan for the next uh couple of month. And finally I I will intr i introduce some related work, yeah. So the big question that I'm concern here is what's the relationship between utterances in multi-party meetings, or how do we analyse discourse structure right above utterances instead of uh those high level intentional structure of discourse? So the concern here is about the very local low level utterance-to-utterance relations in multi-party meetings. Uh this is the local discourse structure. So the uh building block of that kind of uh structure would be the uh utterance and possibly one of its properties, the dialogue act which we're currently uh working on or uh and uh for meetings, yeah, so I've mentioned uh uh before that meetings is a bit different from uh monologue and uh dialogues. So uh in meetings I uh from my uh reading of the one of the AMI meetings I_S_ one O_ eight, uh the uh the four meetings, I noticed that uh uh the interaction, degrees of interaction in the meetings uh varies from parts to parts. So for some part uh the interaction is very strong. So it's obviously a dialogue. But uh for some part part where the interaction is very weak, uh just uh like the uh chairman or project manager is mm uh coordinating the meeting. He's uh pushing the meeting forward, then this kind of s organ organisational structure or part. Uh uh and also other pr uh parts like uh the back-channels without any uh semantic content uh from other participants. So this kind of uh interaction might be better analysed as monologue, which uh in which there is interaction, I suppose. So Uh-huh. Yeah. Yeah. Yeah, yeah. So yeah, so this is going to be based on the utterance to utterance. So for this utterance, the next utterance, if it's from another pa uh speaker, then uh if it's not a back-channel or uh something similar, then it's a dialogue. So if speak changes. So yeah, speaker change is Yeah. Yeah, yeah, b but uh sh short uh turns uh like mm or something very similar might be the uh seg second type of interactivity, the weak interactivity. That might not be analysed as a uh monologue, but as a dial uh no no no, wrong, uh not as dialogue but as um a monologue. Yeah. So for that part uh for that kind of back-channels we can just skip that and uh uh to not treat them as interruptions or something like. Then yeah, we could just uh Yeah. Yeah. Speak change speak change would be yeah. If if it's uh just some kind of back-channel or it's just a signal signal the s the speaker is listening or is understanding what's going on, yeah. Yeah, yeah. Mm-hmm. Uh-huh. Um Yeah, so Uh yeah. Uh so for interactivity this is the only slide. So yeah activity is uh I'd define interactivity to be as a degrees of interaction in meetings. So which might uh speak change whi uh uh would be a very good indicator, yeah. Mm-hmm. Yeah, yeah yeah. No, no. Yeah. No yeah. So actually, so uh this is a very rough idea of interactivity. If we go further, we might distinguish uh the interactivity between two, f three, or all the groups. So that might be some final definition for the uh interactivity, yeah. Yeah, so that might be the second class, yeah. Weak this is a kind of weak interactivity. Yeah. Yeah. Yeah, yeah, yeah, yeah, yeah, yeah, yeah, definitely, yeah. Yeah, yeah, yeah, that's also true. Yeah. S yeah, yeah, actually 'cause uh I've looked uh Yeah. Mm-hmm. Okay. So for that case, yeah uh yeah, so this might be a weak definition of interactivity. So if we could figure out uh if they are really interacti uh interacting with each other, or just uh uh incremental mm presentation of some information. So that's d uh different. We might um yeah need a more strict or finer uh definition of interactivity. Yeah, ju uh for this use of the uh for the use of this term for the indicator of the uh degrees of interaction, uh I think for for the current uh proposal I think that it's just for distinguish between dialogues and monologues. 'Cause for these two types of discourse, uh the uh the the cons the the two that we are going to analyse the discourse structure might be d different. Yeah, yeah. Yeah, yeah. And also 'cause uh I have uh read some of the M_O_T_A_ uh corpus. Uh so for some part of th that kind of meetings um uh several utterances or dialogue acts are just uh tagged as statement. So that I don't think that would be very informative for high level str analysis. So that's all what uh that's why why uh I'm distinguishing the two kinds of discourse here. Mm-hmm. Mm-hmm. Okay. Mm-hmm. Yeah, the like yeah. Yesterday, yeah. Mm-hmm. Might be a Yeah. Yeah. Yeah, yeah. Yeah. Mm-hmm. No, just a uh turn exchange, yeah. Mm-hmm. Yeah. Actually sh uh she used some a a a bunch of features to do the automatic parsing. Mm-hmm. Mm-hmm. Uh yeah. Utterance to utterance, yeah. Very low level, yeah. You cou yeah. Yeah, i to uh just we need to see uh how well we could work out of the utterance to utterance. Yeah. Yeah. So uh yeah. Mm-hmm. Yeah. No. Hmm, hmm. Mm-hmm. Yeah. Yeah, yeah. Yeah. Yeah. Mm-hmm. Yeah, yeah, yeah. So yeah, for interactivity this is a very weak definition, yeah. If we need more then we might use some more finer uh f finer uh definition for interactivity. Yeah. Yeah. Okay. Yeah. Yeah, okay. Mm-hmm. Yeah, okay. So mm so for dialogue, yeah, mm there would be uh some kind of uh interaction structure. So it for uh in previous discourse studies, uh especially in discourse analysis exchange uh is used to describe this kind of uh local structure. And in conversation the analysis so adjacency pairs are used to describe this kind of structure. Uh but uh in these two strands or work they uh just come up with a list, maybe not uh exhaustive. So uh I'm been think about uh if we could come up with some kind of more principal scheme uh taking some of the uh information in the uh uh uh in the uh interaction uh into account. Like uh is the small part of interaction is is exchanging some information. Uh here information is a little vague. It could be some kind of knowledge what the uh speaker knows or or some kind of thought what the speaker is thinking. So these two kind of uh inform mm two kinds of the information uh might be some kind of a difference. So this is a little bit like the Bell's mm exchange of information uh for the knowledge. If exchange of uh opinion uh uh some kind of so it's uh very related to thought, of thinking, yeah. Uh in some case it s differs 'cause uh when we exchange knowledge, we might not challenge uh the speaker, but uh when the speaker is uh pre uh is uh talk about what he feels, what he thinks, is proposing something or s uh then other speakers might might chan challenge him or her. So that kind of structu Yeah. Yeah. Yeah. But uh that might b be some difference inter interaction par patterns. Might be. 'Cause uh for information if we uh somebody's uh seeking some information, then he uh ask some question and uh the one knows will uh gives gives the answer. But uh uh for the uh opinion or thought uh stuff, then the speaker might just uh uh present some idea and others will comment or uh challenge uh the idea. Mm-hmm. Okay. Mm-hmm. Yeah. Yeah, it's vague, yeah. Yeah, yeah, yeah. Mm see 'cause uh so far I'm s still looking for a uh so-called principal scheme for the interaction uh for the interaction. So Yeah, so Yeah. Yeah. Yeah, so yeah, for uh p uh the discourse analysis they're using exchange, but uh they have no uh no classification just to come up with okay, so uh you mean uh uh Sinclair and Coulthard. That is. Also at Geneva school they I don't think they list some kind of exchange. But uh the uh exchange structure uh and uh yeah, exchange is first uh developed in uh systemic functional grammar along the uh uh Halliday and and Martin's work. Yeah. So uh actually they they have some something in in a textbook about the exchange. Uh they call it negotiation. So they have uh some flow charts of the uh uh detailed analysis of uh uh exchange structure, yeah. Yeah. So um uh I don't think there is any kind of principle classification of exchanges. This is i yeah, w Mm-hmm. Mm-hmm. Okay. Mm. Yeah, I_R_F_ uh kind of structure. Yeah, yeah, yeah. Okay. Mm 'cause uh from my previous experience we see ananys analysis of information seeking dialogues over telephones. So it's um uh about the tourism information. So from uh um my analysis of that kind of dialogues I could say that there's only there only four basic uh exchanges depend on the information uh direction of the information flow, and also the speaker role. 'Cause in the information seeking dialogues the uh t roles of the two participants are different. One is mainly the uh from the uh agency, a tourism agency. That is main uh uh information provider uh while the customer is the information seeker mainly. Yeah, that's uh so theref uh and also mm uh the direction of information flows for some kind of information, like uh the uh uh price of a specific route, uh the spots they're going to uh visit, and also on the uh kinds of uh ve uh vehicle or or transportation, yeah. Uh-huh. Yeah, yeah, yeah. Yeah, yeah, yeah, yeah, yeah. Yeah yeah, s yeah, yeah. So but there also some variations which I call complex exchanges. So that's uh that's uh just a list. Mm not like the uh basic exchange uh exchanges, yeah. Yeah, yeah. Yeah yeah, parametri yeah, I I I'm looking for parametric, yeah, scheme. Yeah yeah yeah. That's uh y yeah. A parame parameter or principle. Yeah, some something like. Yeah. Yeah. And also yeah. A and also s in 'cause in the meetings uh the speakers might might report something uh they've done in the past and uh uh uh uh talk about th what they are currently doing and also planning some future actions. So this kind of information might also be helpful. Mm-hmm. Mm-hmm. Yeah. Yeah, yeah. Mm-hmm. Yeah. Uh not marking. Just uh for my uh reading of the uh I_ I_S_ one O_ eight uh four meeting series. Yeah, just to reading this part, yeah. Mm Mm-hmm. Yeah. Okay. So this is for the uh dialogue part. For monologues uh this is uh yeah. Okay, so no further uh information is here. Uh Uh Mm 'cause uh so for the information and action the uh the action here is not dialogue act but uh some kind of uh uh physical action uh that uh the uh participant's uh i i is going to uh or has done in the past. He's going to do the future. So Uh not Physical sense, yeah. Physical sense, yeah. Yeah, so uh this kind of might be the uh some kind of semantics of the utterance or or exchange. So uh this is just one um thing I one parameter uh for mm-hmm. Uh I mean uh here the scheme is for exchange scheme. So yeah, for uh for for uh exchange of information. So there might be the uh s speaker, and he uh a hearer. So who speaks to whom and uh uh S so this is the uh uh so direction of the information flow and also so speaker role might be first uh parameters for the uh ex uh information exchanges. So um but for action that might be uh some kind of uh propose or accept or reject kind of uh uh exchange structure, I mean. Uh-huh. Uh no no. No that's uh that's the uh So we're going to do something and uh uh so could you do something or this kind of uh action related utterance I mean. Yeah. Okay, technical sense. Yeah. Yeah. Yeah, yeah, yeah, yeah. Okay, yeah, yeah. Okay, yeah, that's n Hmm. Before come up with the scheme, you mean. Yeah. Exch So yeah. Uh yeah. S Stent yeah Stent might does um yeah. Stent, yeah. Mm no. Uh so so 'cause I'm I might not familiar with these na Mm-hmm. So uh is it the work uh described in the c uh in in uh in the coconut project or something similar? Yeah, yeah, yeah, yeah. Y yeah, look uh not very much detail, yeah. Yeah. No, not. Okay. Yeah. Yes, yes. Yeah, I know okay. Oh. Uh I know this guy, yeah, yeah. Uh, Jim @'s student. Yeah, I know. No no, in Saa Saarbruecken. Yeah, yeah. Okay. Yeah yeah yeah. Okay. Yeah, I know, her. Okay. No. Mm-hmm. No. Mm-hmm. Yeah. Yeah. So main actions I guess might be some some high level uh problem solving or decision making uh uh concepts, uh not like the exchange which which I I would like to be very low level utterance to utterance relations. Uh I mean uh I mean the m main actions might be some it might be, yeah, built built Okay, so well w Okay. Okay. Okay. Physical. Some kind of physical actions, yeah. Okay. Yeah, different scheme, yeah. So yeah uh th the the main actions that uh you're talk about I think might be, yeah, built up from the exchange, yeah. Ex at least exchange might be uh one kind of uh important information for that kind of Yeah. Yeah, okay. Next. Yeah. So yeah, for the monologue I've I have not uh looked into the details about the this kind of uh relations for the monologue parts. Just a list of these uh relations. And uh so uh next I need to uh review the literature and uh come up with s schemes for the real data annotation. And after that uh we w 'cause this mm is not in the hub corpus. So we are not going to get l all the meetings annotated with that kind of. So uh so we we might ju first try a few meetings and uh have a reliability study. If it's reliable enough, then we must just uh just annotate a small part of the meetings and using some learning techniques to boot boot-strap the uh uh automatic uh analyser. Mm-hmm. Yeah, if uh if it's based on the uh um dialogue acts, yeah. Mm-hmm. Hmm Yes. Hmm. Or fits the meeting browser. Yeah. Okay, so mm This might not be the final goal that uh that could be fit into the uh browser. But it uh might be some kind of uh uh information which might benefit meeting actions or some more high level decision making or problem solving. 'Cause this is the building blocks, I guess, yeah. So Mm-hmm. Yeah. Okay. Mm-hmm. Yeah, yeah. So after this if uh if uh uh we might do some mm pe uh from here we might be some uh uh, yeah, c problem solving, decision making process to track that kind of information. Yeah. Yeah. Messy. Yeah, yeah. Hmm. Mm-hmm. Ye Yeah. Yeah. Mm-hmm. Yeah, ma ma uh s Mm-hmm. Yeah, that's kind of uh Yeah, yeah, yeah. Different feature combination, yeah. Yeah, yes. Uh uh yeah. Uh I mean I was supposed to chunk uh some part of the AMI meetings, you know, to see if the chunker channel switchboard works well enough or not. If not, then mm we have to use some kind of semi-superv supervised learning or some uh boot-strapping to to to do the chunking. No, no, 'cause I was a bit worried about uh this kind of uh discourse structure, yeah. Yeah. Yeah. Okay. Uh dialogue segmentaton. Yeah, okay. Mm-hmm. Mm-hmm. Yeah. Or AMI data. N no. Okay. So Mm-hmm. Yeah. Mm-hmm. Yeah. Mm. Yeah, yeah, for Switchboard, yeah. Direct chunking, 'cause we are not going to in a tree out of the AMI data. Uh so th uh we uh D_F_K_I_, 'cause they're working on some H_P_S_G_ parsing. Yeah, might be. Might be, yeah. If the Uh if uh I'm I guess that if they're different then we might just uh write another script to do the translation. Mm-hmm. Okay. Yeah, yeah. I need to look at some of to see. Mm-hmm. Yeah, yeah. Okay. Mm-hmm. Mm-hmm. Twenty minute, yeah. Yeah? T_N_O_, yeah. Okay. Yeah, 'cause yeah they did some work bio project with the H_P_S_G_ parsing. At least bio. Yeah, well maybe. Okay. Yeah. Okay. Mm-hmm. Mm-hmm. No. Okay, okay. Mm-hmm. Yeah. Uh not sure c uh Okay. Yeah. Yeah. Okay. Hmm hmm. Okay. Yeah, yeah. Yeah. 'Kay. Yeah, yeah. Hmm. Yeah, yeah. Okay. S so then later, yeah? Okay. Okay. Yeah, it's yeah. Okay. Sorry? You mean mm. I will try my best, yeah. Oh yeah. Uh I m I might yeah, 'cause uh Julie and the boy is here. So. it's a little tiring 'cause my parents have left. Uh yeah. So I might take two weeks off two weeks off in uh May, yeah, sometimes, yeah. Off, yeah. Okay, thanks. Uh yeah, hope I hope so very soon. Yeah. Yeah. Okay, thanks.
Five years just because of the floods bringing the ceilings down. So On its great joy. I think they probably ought to, but uh they well, they've only just started realising that treating this as a legal issue is probably the right way to look at it. So they're not really the sort of people that are up for understanding um the intricacies of the committee, I guess. I mean i you would have thought they could, but it's r it's actually quite hard work. The law isn't very clear. That's a good question. They p they uh they might be able to. It's just it's so strapped now, um you know, like the Citizens Advice Bureau is so hard to actually get access to because they've they have so many cases and so few people. So But yeah, that's probably the next thing they'll have to do. M Mm-hmm. Okay. Yeah. Right. Well um I thought the right thing to do was to review what Wei-Xun's been doing in the area of dialogue structure for work package one over the last period. So basically for him to talk about what he what progress he's been making and then for us to um for hi for him to describe what work he intends over the next period of the project, and for us to figure out what help he needs with that. Um m maybe give him some help figuring out what publications he's gonna um get out of this and for us to make sure that it fits into the overall project. So um this is something that you know, we're not gonna do everything today, but I wanted to make sure that we sort of all aligned on what he was doing at the moment and and then sort of figured out how to give him the best advice coming up. So that's that's why I called this. I'm a little bit confused. So um when you say interactivity, do you mean um well, how would you characterise a highly interactive meeting? What are the measurements you'd apply? So you I think what you're doing is segmenting meetings into highly interactive parts and not so interactive parts, where you think some of them are like a bit like dialogue and some are a bit like monologue. So what measurements would you apply to um find out which is which? So you mean relatively short turns and speaker change. S So so am I right in thinking you think that you can segment meetings maybe into two t types of areas, interactive and non-interactive? So what are th what are the characteristics of the interactive areas? So w so okay, so but you just said, I think, that you would characterise short turns as being the non-interactive parts. Oh okay, well we we can discount back-channels. I mean uh c are you is it okay if we talk about this for a minute 'cause I'm confused. Or or do you get a c Alright. Alright. 'Cause if this is all we've got, I think we should uh have interactive and non-interactive on the board and then figure out what properties going to each just so we're clear what the two mean. Uh Well uh okay But are you you need to be more precise what you mean by speaker change. So y do you mean all of the speakers are involved in it? Is it sufficient for two speakers? Right, and then Okay, but that's n you can that's definitional, right. So so I mean you wouldn't necessarily consider that to change the turns. Those are just back-channels. So one speaker will have the floor channel um throughout that and so I guess what you're saying is that the interactive sections would have shorter speaker turns when you're not considering the back-channels as interruptions. Um so Well do y Yeah. Sufficient but not necessary maybe. It depend if you had so a bunch of people all just giving information and opinion around the table and they never actually ask questions of each other, but the turns were actually quite short, which one would it be? Mm-hmm. Right, okay. I remember you saying this before. So basically you want to do the segmentation because you're going to apply completely different analyses to the two sections. Um Right. So Wei-Xun quite takes quite a strong position on um this um difference between these two um kinds of group activities because he says that they admit different kinds of analyses. Right, the with the non-interactive, more monolo mono-logic sections of the meeting um you have to uh analyse them using something akin to rhetorical structure or something. And uh with the other sections you uh um you you use some other theory to look at them. And I actually uh think that that's a slightly extreme um position um with regard to the literature that a lot of people might say it doesn't really matter who the different speakers are in an interactive section. You can still um build maybe the the same argument structure over the top of the basic um acts and propositional content that's there. And uh what you get is a messier version of the argument structure because not everybody has the same argument in their brain, you know, for this. But um but that the same theory or the the same analysis can be applied to both parts. So I think it's important to keep um that in mind, and if you if you do need to argue that these two kinds of group activities uh can can only be analysed if you use different analytical mechanisms, then you have to explain why that is. Right, so it it puts an extra burden of proof on the work that you do. Mm. Right. But to the grou degree that a group is building an argument together, I mean it I think wh what you're doing in a group discussion isn't necessarily one coherent argument f for the person, but an argument that is the group view. And so it's maybe it's not R_S_T_, um but it's um more akin to um I don't know, is it Toolman? But, you know, not that i but you know, the kinds of things where people to try to represent argument structure. Are are the important aspects of it for this analysis really that substantively different though? But i if somebody if there's been an argument so far in a group meeting, and um somebody makes a statement, they have an intention about how to modify that group argument. Right, but but you're Okay, so the r the relation names aren't aren't right, but um Right. Okay. Well the relation names are less important. I mean people have argued for ages about exactly what relations were in any of these theories. And that's that's less important than what the theory is trying to achieve. And it just seems to me that if you've gotta have two dif separate apparatuses for the um speaker internal um argument structure and the group argument structure, that that's more appara apparatus than you should actually need for um from doing this kind of analysis. 'Cause in both cases you're um building up an argument publicly in this group meeting and then, you know, adding something or changing the existing argument in some way by relating a contribution to what's um already on the table. So Mm. Yeah, c I just think well, I mean it all of these things when they encounter a new genre of um of uh actual data, then they end up modifying building new kinds of relations with new kinds of semantics and adding them. That's the entire history of these. Mm-hmm. Mm-hmm, yeah. Yeah. Right. So there you've got somebody who's doing theoretical development that would claim that you don't need two separate an analytical frameworks for these two types of activities that Wei-Xun has identified. Yeah. Mm-hmm. Yeah, I just prefer it to be one thing rather than two completely separate things and then the added explanation of why it's not possible to have the same thing applied to both halves. Hmm, yeah. Mm-hmm. Mm-hmm. Yeah, which wouldn't probably wouldn't give us the kind of um information that we need about relations across turns, just because we really are critically interested in the argument um that the group is building up so that we can find the decisions that they made and um any p uh counter proposals to proposals that have been made and things. I mean it really is more in the spirit uh of the R_S_T_ ish relations than um than in the damsel kinds of relationships. What we want uh out of the c um the relationships between speakers here. They are jointly building up an argument structure, and that is the critical thing that we're missing in order to be able to um understand what's happening in these meetings. So we wouldn't we wouldn't wanna do the same thing as what um she did in this case, bec just because um damsel doesn't give you the right sort of representation between speakers. Oh, does it? Okay, um I haven't seen it on that this kind of data. So maybe Right, okay, but are what I mean is, you kno wanna make sure you get the argumentative relations, however you do it. Whichever relations you have to build, you know. Not I'm not saying I'm not saying that there's a particular set of tags or relations that have been defined out there that are the right ones. I'm just saying we need the correct ones for th the structure that we're trying to get out of this. 'Cause we need certain information out of this analysis. That Yeah. Well Wei-Xun's been looking at the data. So that's that's a step for this. Mm-hmm. Mm-hmm. Mm-hmm, yeah. Well, what you'll need in the end is something operational. So it's something that you can use to actually segment meetings into the um the areas that you think are important for this analysis, right? So um it's not quite clear to me what the different kinds of things are. So it would be I think handy if you could take a meeting at some point and say, you know, this is a interactive area and this is a mono-logic area because and we could come up with what kinds of measurements you need. Um I mean we have w we haven't done past work here on segmenting meetings in that way, but we have done work on characterising different kinds of meetings um in terms of how highly interactive they are. But it's a different definition, right. It's in terms of how much all of the people involved in the meeting um take some take equal part in it as opposed to um like the highest state is people in the meeting just driving it and and uh the other people following whatever the high state is people want. So um but those kinds of measures um, you know, we could use to um to do this kind of segmentation if it's important to the analysis, right. W but we need clearer definitions of what the different things are, how many different kinds of of um activities are there. You know, is it just interactive and non-interactive, or is there are there there more grades of shading than that. And um, you know, what are the things that you expect in each. What's the back-channelling behaviour. What's the speaker turn, um you know, what kind of dialogue acts do you expect. I think that would help you pin down what the definitions are. Do you think it matters whether something is knowledge or opinion in the way it contributes to the group process? Yeah. I I sort of have always taken the view that um that well, people aren't very good at at this kind of rationality, you know. And the difference between information and opinion isn't that prominent in the way people do their reasoning. So um Yeah. Well you you can't define 'em that way. You can't define the difference between information and opinion that way because that's um basically defining as the what happened about it. And I think what tends to happen is that um if people challenge the the facts, you often discover that they weren't actually solid facts anyway, you know, that they're I think the boundary between these is kind of fluid. So I guess the question is is it important to what what's it important to? So do you need the difference between information and opinion to do this analysis? So what have you been looking at? So you're you've got books that high on your desk. Which ones are they? Mm-hmm. Mm-hmm. Mm. Okay. So y um I'm sort of getting that you think there is a lack in the uh theory to do with exchange structure at the moment. But I'm but I'm not quite clear yet what you think the lack is. So Um the well are you thinking it's been a while since I've read this literature. But I mean my understanding of exchange is is that the initiating move introduces a goal to the discourse, and then the remaining moves um will uh contribute to either the um the meeting of that discourse goal or the decision not to bother with that discourse goal, right. And so um I'd be surprised if there wasn't work describing things in those terms. I mean I know Sinclair and Coulthard is is basically about teaching yeah. So you wouldn't expect that to necessarily um apply uh in in wider contacts in a way. Like you might expect to have to extend the theory to do that. But I th I think there are other other people have looked at exchange structure and described it in these terms in a way that should really suit your needs. Mm-hmm. Yeah. Mm-hmm. Mm-hmm. Yeah. So this is a l a lot like George Horton's work on saying what are the different parameters for um these exchanges. You know, what are the things that characterise the set of exchanges that can happen? Mm. So so is your complaint less about um less about knowing what makes an exchange and more about knowing how you parameterise exchanges for this genre? So what's the range? okay, so you're worried about what's the range of exchange types for this genre, and that's what you think hasn't been done. Uh-huh. Okay. That's clearer. Mm-hmm. Yeah. So so have have you tried I mean um I know of one previous attempt to establish what exchanges are needed for f um some kinds of industry meetings, and our experience in that project was that um you can't really identify exchange structures in these meetings because they they don't really follow that pattern. That exchanges are clear in um well, i there're clearer in when you've got fewer people. So there's fewer people trying to pull the discourse goals in different directions. Um and they're clearer when um your goals are quite limited. So when you've just got information exchange and you know, the or there's a clear task that the people are trying to do and they're focussed on that task, as is normal in dialogue elicitation like map task or like your like your informa which like your information seeking dialogues, which wasn't dialogue elicitation, but it was very constrained in genre. So um the the answer to the question might be nobody knows what the catalogue of exchanges is for real meetings because um or for these decision making meetings because you can't identify exchanges, right. So have you gone through the meetings and looked for initiations and responses with without a prior catalogue, just knowing that init initiation in introduces a new discourse goal and a response um meets or negates that goal in some way. So have you tried marking these exchanges? Mm-hmm. Yeah. But but a reading of this, you know, th could tell you that there is no case where you're gonna be looking for initiation and responses that's gonna work, you know. 'Cause you could go through and see that um actually the pattern of of addressing different discourse goals is too dispersed to um to be able to do that kind of mark-up. But I g yeah, I mean i obviously you wanna have a front-runner scheme i in mind before you So that's it. Yeah. Do you mean an action in terms of what the group is requiring him to do or do you mean an action in like picking something up? So action in the technical meeting sense or just action in the physical sense. Right, okay. Oh, so that's not a physical action, right? Uh uh if I pick this up, which is a physical action, you can't reject that. What do you mean? Right. That's right, that's a action in the sort of technical sense of meetings. So you are actioned to do something, which is part of the permanent record that yeah, okay. Well, there are very few uh schemes for marking up exchanges on actual data, right, because um people haven't considered it very important. Well I always thought they get tired after marking up the dialogue moves so they nev never bother with any higher level dialogue structure. So we did it in the map task, but I don't actually know of other cases where people bothered marking anyth anything higher level than the map than the move structure. Right? You don't know of On yeah, I've s uh sorry. Um but it c it's not been very popular car probably 'cause it makes um Different. Mm-hmm, yeah. Well the first step is find out in the what how much of the material does the collaborative problem solving cover? So how much of a genre clash is there here? Well it's Yeah. B th But there there are people in the field that are moving that direction, like Martin. And also what's important here is the theoretical clarity about what an initiation and a response is. So what is it that ties um different acts together at a higher level, you know. Mm-hmm. But I think more important than looking at um whether parts of the meeting are highly interactive or not so interactive is uh looking at what sort of task the people are trying to do at the moment. So you know, are they doing um uh i are are they uh i as in the kick off meeting just exchanging information to get started with a project or are they uh actually addressing some specific design um criteria at the moment. Should try to decide which shape the thing is. Um because that's going to affect the kinds of relations that you get between the utterances. And so that is much more like um the um the literature from McGrath about the different kinds of activities groups get into. So I think that's another thing that has to be pulled into this because um you you do require different uh uh different kinds of relations. Or you expect different kinds of relations from some over-arching set in um those different kinds of activities. Uh Um so what's the high level version again? P s Right, okay. I mean i the meeting actions is just gonna be a segmentation. It's not going to present any s any micro-structure. So it's just gonna say from here to here they're doing something. That's all that that scheme is gonna do. W It's n That's not no, those well, that's individual actions. And the meeting actions are meant to b uh become more um semantic and more theoretical than the kind IDIAP have traditionally used. So so we need to still preserve some uh sense of the activity. But rather than saying it is somebody's at the white-board, it'll be, you know, that somebody's explaining something that the others are focussed on. Yeah. It's it's a different scheme. Yeah. Mm-hmm. Well you might need the meeting action segmentation in order to know where what uh what area you're circumventing for so, it's just exchange structure in this area that you're looking at, and then that's a solid boundary between the next bit or something. I don't know. Um but w well, time's getting on. So, you know, what practical things can we do to help this work move forward? So y show what else you're you're thinking about, yeah. Mm-hmm. Mm-hmm. Mm-hmm.. Well m marking up exchange structure if if it's there and if um if you've got a working scheme for it is relatively cheap. It's a lot cheaper than doing dialogue act coding in the first place. Um right, well one presumes it is, otherwise it's gonna be ra radically expensive to do any kind of mark-up. So um the you know, it's something that we can look at, but you need to what we need it's more important than knowing, you know, that we need exchange structure marked up on X_ amount of data at this point is knowing um where you're headed with this, I think. So what's the good that that uh the project gets out of this in the end. So you know, what's the w what's the headline results of the work on the project, I think is is what we're trying to get a sense of here. So you get your exchange structure marked up. And say we just put in the money, get exchange structure marked up on that entire hub or, you know, figure out how to boot-strap it so we we get this um uh how do you how do you sell that as a as a project goal, you know? 'Cause people are gonna say exchange structure, so what. What do wh what do you what how do you plug that into the rest the stuff, right? Yeah. Mm-hmm. Yeah. See 'cause I think that um just to make this m more efficient, you should be thinking about the high-level goal in order to focus the work that you're doing rather than thinking well, you know, there's these theories, you know, how can we apply these theories to it. Um and then seeing what comes out of that that might be useful for the meeting browser, actually trying to say yeah, we have a goal of being able to do this with the meeting browser, um what kinds of dialogue structure would we need to be able to do that. I mean uh I don't know, maybe you'd wanna disagree with that, but I this more goal directed might actually make it easier to um bring these things to publication in the long run, right? Right. It's not necessarily what they say. But it's something at least identify something that we'd hope to get out of it, you know. Mm-hmm. Yeah. I It just helps you know which theory you should be concentrating on. So you know, that immediately discounts exchange um structures that are mostly for information seeking alone because the notion exchange is to restrict it there, and in favour of um these these more argumentative things. So I mean uh I do think you should keep in mind what the final goal is so that you can plug this into something at the end and and um show some benefit of having the mark-up. Mm-hmm. Yeah. Yeah. I mean operationally it's not gonna much difference whether it's hub or not, right, because we've got the bank of people. We have to get the hub stuff through, but um you know, he's n not ready with an annotation scheme that can be used at the moment anyway. So And it it's not that expensive. I mean it's something that we can afford off budget 'cause Yeah, it uh and it's too different a genre, yeah. Mm. Mm-hmm. Yeah, that's not help either. But yeah, they're sure to do that. Mm-hmm. Mm-hmm. In the m Mm-hmm. Y Um well, I don't remember that about the w I mean the Oh, but that was about what task the person doing the browsing was gonna do. It w they didn't say anything about what sort of information they'd want from the meeting itself. It w it was about uh i That was the that we the browser task. You know they've identified these seven different tasks people might wanna do with browsers, um you know, like uh join a meeting late and ch um catch up on what's happened in the meeting so far, be at home waiting for the washing repair guy. And Yeah. Okay, well c maybe. Okay. Mm-hmm. But I thought you wanted to avoid doing stuff that was s stuck in this particular domain if possible. But you know, th they weren't concentrating Well yeah. But that doesn't mean you have to be um using the techniques that you can be using open-domain techn Mm-hmm. And you don't think that can be done in open domain at all. 'Cause we don't wanna get into um working with ontologies. I mean one of the strengths of this grouping was that, yeah, they all work with the closed domain ontologies and we'll do the open-domain side of things. So I don't know. We'll see. Oh right, yeah. So we don't know what happened at that one. Yeah. Mm-hmm. Yeah. So you're advising Wei-Xun that he should be using like any ontologies and propositional content that um D_F_K_I_ and T_N_O_ bank in the C_V_S_ repository, or or not? That's what I'm not clear on. With our without. Mm-hmm. Okay. Can actually try. Mm-hmm. Yeah. I don't know when they'll bank these by the way. Um 'cause that's it's not hub. So i it's one of these things where everybody's supposed to bank as many things as possible. So you know w Yeah. But open. Yeah, it's not really an open domain task, yeah. Well eventually the hub is supposed to be forty percent non-scenario. But that data's definitely gonna be kind of coming in the last. So Mm-hmm. Yeah. That's quite reasonable. W we've got five ISCA meetings actually which are still uh an elicitation task, but they're not our domain. So that should help some. And they've been declared hub. So so they'll accrue all of the normal an annotations. Of course yeah, you'll you'll never get the T_N_O_ and D_F_K_I_ stuff on it 'cause they're only billing stuff for the one to make the Yeah, just figure out how we're gonna help him to to try to make this more efficient. Yeah. Yeah, but we should make sure we finish that off, because otherwise that work's just wasted, right. We're gonna use chunks for something useful. Yeah, chunks are gonna be named entities if nothing else. Automatic segmentation at which level? Uh-huh. Dialogue act, yeah. Yeah. Yeah, and that's a problem. Yeah. Certainly if if you've got chunks it ought to help you know where your dialogue act boundaries better not be. Yeah. Mm-hmm. And it may be even more mechanical than that based on the um examples we're seeing from D_F_K_I_. 'Cause they're they're hand marking stuff and the dialogue act boundaries are supposed to be intentional. Um but they seem to always be marking them that um uh after verbs, you know, with clause m uh it's very it looks quite mechanical. So we're querying them about this because that's not the way they're supposed to be doing it. And Well hold on. But when do we need chunking by? And it you know, so y I don't like the idea that, you know, you start something and then eighteen months later you pick up to do the last ten percent of the work to finish it off or whatever. So it I like things kinda tidy. Right. So how much work is there left and when do we actually need it by would be useful to discuss. The AMI data is is there. There's I mean there's hub set A_ is transcribed I think um in final form. You we can check the Wikis. But you know, the data is rolling in. So And but Barbara still says somebody's done this for chunks. I'll have to look up, you know, so Mm. Hand-chunked. Mm-hmm. But that shouldn't be that hard to do, right? How much data do you need? I don't read lips. What was that? Was it hard? Yeah. They the data how much data you need was not a problem with the Switchboard 'cause he had lots. So he didn't have to think about how much. So we need to know how much data you need hand-chunked. And then uh how do you hand-chunk stuff, right. So you were were working on instructions for doing hand-chunking. For Switchboard, yeah. But then he there were two possible options for uh for doing this on the AMI data. You could either um essentially go the same route as they did with the Switchboard um, you know, do the Penn tree-bank and then create the chunks off that, or you could chunk directly. And it wasn't clear to us which one of those two methods was the right way to do this. Right, well, so you'd have to produce as part of this exercise. You would've thought so, but I dunno. Oh yes, yeah, I'd forgotten about that. But that's not very similar. Well, I don't know very much about grammar. Is th is that similar enough that that this same method could apply that we derive chunks off the H_P_S_G_ output? I think you might lose a lot in, you know, this step. So basically your your chunker is trained on something that was derived off Penn tree-bank. And it might be different enough that that actually causes it not to match. You know, if you're suddenly evaluating against chunks that were done using H_P_S_G_ I don't know about grammar, but it you know it Yeah. And the mappings are can be cr pretty difficult for this, right? Or is is easy to map between finer grained distinctions and coarser ones? Think so? Yeah. Huh. The names would suggest that this was the idea, right. Yeah. Right. So y that sort of assumes that we think it's better to do this by deriving chunks from um some parsed format than just hand chunking the stuff. Mm-hmm. Yeah. So let's try it this way, and then that that puts off the question of um when do we have to do this to when we get H_P_S_G_ stuff, right? So can you ask um D_F_K_I_ whether they can give us a couple AMI meetings run through their H_P_S_G_ process? I don't even know what time scale they had in mind. I mean it is this something they w they were doing it at the moment, but I don't know whether this was very preliminary or do you remember? Twente, yeah. Yeah. Y you're expecting me to remember stuff, aren't you? Oh yeah. Oh, I don't even know which one I've missed. Brussels, Amsterdam. T_N_O_, okay. See, I wasn't there. So I don't know where it was. Um Yeah. Yeah. So ask 'em, right. So it's talk to Alex, see if it looks like it maps by looking at the Verbmobil. If it does, ask D_F_K_I_ either for data or a time scale, and you're gonna need to know how much data you need in order to do that. So do you have a technique for estimating how much data you'd need? I don't know how you do that, so who knows how you do that? Okay. So just ask D_F_K_I_ how much data and when, right. If Yeah, I know. Uh we don't have any particular time in mind when we need a chunker that works on this stuff. So Yeah. Yeah. Well I gue if we did a chunked version of the AMI data and stuck it into C_V_S_ under the automatic contributions then um people would end up using it, 'cause they do that, right. So the question is at what stage should we put that in? Yeah, so post-evaluation so we can put in some idea of what the evaluation is like. Mm-hmm. No, but he's relying it on it th this summer. Mm-hmm. So when when do you think that would be? When when is she sending somebody? Oh for the next one, mm-hmm. But that's a pretty tight time scale then, right, 'cause it'd be for the fall we'd wanna know the evaluation. Is that the time scale you're thinking of or How it could be done. Yeah. Uh that's all I think. Uh Mm-hmm. I think you should still ask Barbara again, you know, c can you have another think about who might have the chunking stuff. 'Cause she has said several times there must be somebody and she can help with that. And um I think she's just forgotten. So So and then we can evaluate on the ICSI, you know, pretty easily. Yeah, even if it's just a you know, on a Wiki the um the references and then why they're relevant, right. I mean you don't wanna make this really formal, or do you? Yeah, a paragraph each. Yeah. Yeah. But that wou that would help us be a lot more specific ab than than just um you know, listing bullet points on slides cause then we'd know sort of which literature you're thinking of um putting in and we could read the most important argumentative stuff. I can have these things anyway, yeah. Yeah. Mm-hmm. Mm-hmm. They haven't addressed this problem. Yeah. Yeah. I'm gonna have to go very soon because I've got this um this hearing to go to. But uh so the practical actions, you know, what are the things we're gonna go away and do. Yeah. Yeah. So you can do this exercise and then come and talk to me some time next week. I'm not in Monday, but the other days I'll be around. Mm-hmm. Yeah. Yeah. S Yeah, so this this is interesting, but i w I don't think we can afford to l um look at this right now. So why don't you bring it by um next week? And i what I really wanna do concentrate just now is is finding out the how we're gonna help um Wei-Xun with with making progress on this. So we've already got you know, he's gonna um construct a list of the references he thinks is important, and Johanna and I and and you as well are gonna contribute to that. And we can put that on a Wiki or we can do it some other way. I mean you know I'm Wiki mad. So um if we share it, it should go on a Wiki 'cause I can't find anything in my email. But, you know, you don't have to do that. Um and then uh how do we you're actually so what you're doing is taking all of the the the most important references for um finding argument structure that might i eventually be of some use to a meeting browser and trying to look at actual data and apply and and see like which one helps you figure out what you're actually gonna do with this particular data, right. So it seems to me that we should take like the front-runner scheme and the data and you should look at not front-runner scheme but front-runner theory and i see if you can speak to the data, you know, using that kind of theory at some point. And might might be an interesting exercise, right? The update. Mm-hmm. Mm-hmm. Yeah. But that's a good starting point, yeah. That'll be the Exactly, yeah. So you wanna go through basically the user requirements and, you know, maybe think about some user requirements of your own and make a case for, you know, these are the kinds of things that we'd want, and look, you're gonna need um this kind of dialogue or argument structure for it. Um uh uh I'll think about that. Yeah. So we're just trying to get records. I mean I'm sure they're not on the web yet. So Yeah. Mm-hmm, yeah. Yeah. Okay. So so we're gonna go through the requirements like that and did did you think that's that was reasonable to after that to try and um actually, you know, take some theory that seems fairly reasonable and think about the particular data and come and talk? Yeah. I there's a few references missing. Yeah. Yep. So then try front-runner scheme on, well, a front-runner theory. Yeah. But try it informally. Yep, mm-hmm. When do we get together again? 'Cause I really do have to go. I promised these guys I'd be there. Uh-huh. But w Reading lists before that though, so right? So we'll get together in about three weeks. Uh Uh-huh. Four weeks, yeah. Right. I'm gonna head off. C 'cause floods are more important than dialogue structure, I'm afraid.
Yeah, mm-hmm. Hmm. Thank you. No, I don't think so. I d Hmm. Mm-hmm. Yeah. Yes. Hmm. Mm-hmm. Meeting action. Mm. They just mention that they want decision um for issues, action items. I think they are more talking about um user requirement for the meeting than soft text, they want to know what happened happening in a meeting. they say that? Hmm. Mm-hmm.. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Contact. Hmm. Mm-hmm. Hmm. Mm. Mm-hmm. Yeah. Um right now d uh I'm doing I'm trying to do the uh meeting action segmentation. J because uh after talking to Jane um just uh we have different ideas whether uh meeting action will be about the same size as topic segments. So um it would be good to um t right now I'm doing this, um looking at the data and trying to see if I can do the meeting action segmentation first and the scheme, and then do topic segmentation again to see if the boundary actually coincide with each other. Yeah, if if that that do, then maybe we can talk again whether um i it it make sense. Mm-hmm. Alright, yeah. Um I have just started. And um so far um I s think uh it's uh it I think it it's a still um about the s about the same level of segmentation we're talking here. But uh maybe the boundary will not be exactly the same. But uh that that the segment Mm-hmm, right, sure. Yeah. Mm-hmm. Mm, sure. Yes. Mm-hmm. Mm-hmm. Hmm. Hmm. Hmm. Hmm. Mm. I want to see. Something. Sure. Mm-hmm.
Aren't they suing him? Mm-hmm. Can't th can't they get legal aid? Hmm. Mm-hmm. Are we supposed to count to ten or something? We don't have to do all that? Okay. Okay. Thank you. Mm-hmm. Okay, so You're gonna give a presentation for us then. Great. Mm-hmm. Mm-hmm. I was just wondering if you were gonna define this more on the slides to come or if this is as much as you're gonna tell us. I'm just wondering if he's prepared Sure. So so what did you have more on later slides about this? Okay. But we've also established that it's not just speaker change, it because if you have mm-hmm, okay, then Okay. Y yeah, yeah. Right. Yeah. Mm-hmm. Well it might be the case that to be interactive there have to be dialogue acts of some specific type in a turn rather than Like statement, statement, reject, accept, you know, something like that. Mm-hmm. Well, actually I don't quite agree. I mean i for example you can't use R_S_T_ across speakers because the definitions of the relations depend on the intentions of the speaker and there's different speakers have different intentions. So you can't use the theory. It's just fundamentally wrong, it's it was meant as a theory for a one person uh right. So that doesn't mean there aren't other theo so for Yeah. Right, but those are very those are really different um theories that are at are trying to get at different things and it and they admit of m you know, some of them admit of multiple parties and some of them assume you're in one speaker's head. Right, and the Mm-hmm. Yeah, the entire theory is based on intention and the intention of one individual. So R_S_T_ would be a bad theory to use. W So modelling modelling a single contribution by a single speaker, you can within that you can use R_S_T_ if you want to, right. But across speakers you can't. They have no relations for things like reject or agree. They the theory doesn't have them. So the the so it's not the right theory 'cause the theory is the relations plus the notion of nuclearity. So we need a different theory. Now I think Alex um S_D_R_S_ allows both kinds. So you know, within a speaker turn you can have things like explanation and cause and all that. Um but then across speaker turns you can have things like correction or I didn't see any examples of agree or disagree in S_T_R_S_, but I there aren't? I mean there are thirty relations. So we should look at at what they are. But I d I don't Well her the relations have a s D Yes, but so I mean the relation names it's not the names that are important, it's their semantics that are important, right. And they're the semantics of the relations are based on the semantics of the utterances that they're relating as Alex s explained yesterday. Um and so um it is important that these relations have a semantics that's tied to the semantics of the utterances, right. Uh p poss possibly, but few people have tried to, you know, do the kind of um truth conditional semantics that, you know, Asher and Lascarides have done, right. So the theory can handle the kinds of things that happen in meetings, as she explained. The actual computational mechanisms they have currently for building up the trees can't. Now I don't remember, was which speaker you in the c in the example she showed yesterday, was it the case that the representation of the utterances and the relations included which speaker was uh I mean it had the at the l at the leaves they had the well, she didn't show it, but they would have the semantics of the utterances. And then she had relation names and handles. Right. I just I just don't didn't remember seeing in the representation which speaker contributed what. 'Cause she did represent um discourse structures 'cause these were the Verbmobil scheduling dialogues. So the utterances in the in the representation were uttered by two different parties that wasn't the you know, in the in the examples she showed us, there were there was some that was just, you know, one party. It was a monologue, but then there was a snippet of dialogue as well with For this specific set of relations that that they've defined. There might be some higher level structure that you wanna pull out of that. I mean I 'cause you were saying you're just gonna you you're you wanna look at this, you know, very local stuff. But from that you c one would hope you would pull out something high, you know No, I think working from the bottom up sounds great, but I'm just y I mean what you can imagine someone saying well how did they decide to make it blue, and maybe there was a long blow by bl and you don't you kind of wanna abstract away from he said this, she said that, he sai you know, to Anyway, I you know, I mean I I don't think especially uh, you know, if you're not steeped in these theories that it makes a huge difference to you whether we use R_S_T_ or something like S_D_R_S_. But it might make much more, you know, I mean Well I think we should ass I think we should try to find a theory that allows multiple speakers and relations to span over more than one spea you know. Because it is true that I mean if you look at things like damsel and other attempts to come up with annotation schemes for for dialogue acts, there was this notion of forward looking and backward looking. Anyway, they're relational. I mean in some way, you know, they're all relational. Um but you needed But there's a different set of tags in there from the kinds of relations you see within R_S_T_, for example. So if you look at Amanda Stent's thesis and the analysis that she did sort of within a turn she used something like R_S_T_. And then across turns she used damsel, right. Well I Well, it gives you things like accept and reject and Oh yeah. In fact R_S_T_ doesn't give you anything like that. Okay. Right. Right. Right, okay. D so I think it's much more impor I think it would make more sense to go about this problem figuring out what kinds of relationships we want and then finding and, you know, like setting down what the constraints are on the analysis game. He's been looking at the data, okay, s to good. Um and then and then there are lots of different theories of argument. I mean you can also think about the sort of joint problem solving kinds of um approaches. So you know, Clark and Lockbaum and, you know, that that kind of thing. That were designed for dialogue, right, I mean, and can be expanded to more than two persons. Actually the Lockbaum stuff is for groups. Yeah, I was just gonna say it would be useful to know what m But who's they? That's the question. Who? Right. Mm-hmm. Well I mean I think that um w it I don't know uh w it'd probably get to re hear, you know, more of your presentation because I think there are lots of different candidates and you certainly can't go through and mark up every meeting, I mean mark up meetings with every scheme we think of. So I think we have to kind of get an overview of what the candidates are m if there's like two or three that we think are good contenders, then we try to see what works better, right. There are a lot of option there are a lot more options. So so let's see what what else you've got. Oh, so that was it on dialogue, okay. Okay. So uh it's very unclear to me why you think exchange of information is more principal t uh did you I mean what did you have specific schemes in mind there? Or spec mm specific people's work that you um But can you give us an example of a scheme that's based on exchange of information? Okay. Mm-hmm. Okay, so I mean I think that um there are a lot of things that we have to look at before, I mean Yeah. I mean and Well, we're marking up a lot of higher level things in our Okay. So um have you looked at Nate Blaylock's work? So he has things like, you know, adopt um so basically it's to, you know, collaborative problem solving, you know, more than one person involved, and trying to characterise things as um You know, if I I can't remember the I can't remember the names of all the actions. No, so so the Coconut project is a project that I did with Rich Thomasson and Barbara DiGinio. Have you looked at that? Not very clear. No. Though there's a lot of detail in the paper. But you haven't looked at it no in very much detail. Okay, so I think you should, um because it was all these things that are done for joint problem solving and joint action I think are very relevant to look at because these people are solving a problem together, right. So um while these schemes may not have they have some es of the essential components that you need to characterise um joint problem solving. So while th um you may think that they're missing something, then we we could certainly augment them. But I think that they have to be looked at. And so so Nate Blaylock tried to okay, so you know, from Gross and Krauss and Gross and Sidener came all this stuff about shared plans and how you really can't understand, you know, dialogue u unless you understand shared plans. But the actual theoretical um or formalisation of it was incredibly complex. And so um Lockbaum simplified it quite and, you know, Sidener and Rich and Collagen have simplified it quite a bit. Um but Nate Blaylock tried to um, you know, define, you know, what happens after somebody suggests something. Well, you can adopt it, you can well I can't remember you know, okay, but he has uh he has a taxonomy yeah, right. Yeah, yeah. Yeah, exactly, right. Um which we tried to apply to tutoring, but it actually doesn't work very well for tutoring 'cause tutoring's just a totally different ball game. But um it you know, we spent a fair amount of time looking at it and it did strike me that for joint problem solving there w there were some good ideas. And there is, you know, a set of tags and what they mean and, you know, that sort of thing. And then you can also look at, you know, cl you know Clark on joint action. Now he's not gonna give you a set of tags 'cause that's not the kind of guy he is. But um But the overarching idea, you know, I mean I think accounts for a lot of the ways in which um, you know, these other people are thi are thinking. But I mean I think that we we basically have to merge some of this stuff that's been done, you know, on meetings with some of the stuff that's been done on collaborative problem solving 'cause there's elements of each that seem really important. And I think th Because it really seems to me that, you know, if you look at and some of this stuff on exchange is like ancient and you know, they didn't have the ability to mark up lots of data and count things, you know. So I mean all the conversational analysis stuff, I mean those guys never count anything, you know. I mean they're uh ps so they see a pattern and then they think it's kinda cute and then they scour the world for third turn repairs, you know. They don't actually say how often do those happen. I mean they worked I mean I know Manny Schegloff really well and they're not counters. They're purposely not counters. They define themselves as non-counters. Mm-hmm. Yeah, th I mean it it does seem like, you know, there's something that we want from that and then there's something that we want from collaborative problem solving. And if you l if you take the exchange stuff, it's not gonna d quite do everything you want, and if you look at the collaborative problem solving stuff, maybe it won't do every that you want. And so Hmm. Like somebody's writing on the white-board or I mean the w meeting actions they presented at the review seemed really low level to me, like somebody stood up. Okay. Okay. Okay. Okay. I mean I I realise it's important to recognise that, you know, what Wei-Xun got up and went to the white-board. I realise that's important, you know. But I d was kind of surprised that those meeting actions. So Yeah, mm okay. Well I think we should approach it both ways. I mean uh you know, I I d You know, the meeting browser people have a lot of constraints. And so what they end up with saying is most critical for them right now might not be the most interesting thing in the longer term. And so Right. I mean people have said over and over again that they would like to understand how a decision has evolved, right. Um and I think that Well, I don't know if all of this low level stuff would allow you to track that kind of thing, but it might at least be the building blocks of allowing you to track that kind of thing and Yeah, some higher level yeah. So I mean I think it's clear that something at this level needs to be there. I mean and to ask questions, like who disagreed or were there any counter proposals or whatever. Even those kind of questions you need some kind of mark-up like this. Well that brings up something interesting because I think probably what we should do is try to get you know, some of these tasks sort of put on, you know, sort of as part of the official agenda. So being able to answer questions like um were there any counter proposals to the bana you know, banana shape or whatever. Um uh or to some specific, you know, recommendation and things like that. I mean, you know, those, you know, strike me as cl clear things that, you know, people who go to decision making meetings want to know if they've missed one or are arriving late or anything like that. So it wou it seems like we should probably get some of these things on the official agenda and say look, you know, wouldn't it be nice if you could have answers to these questions, or something. Oh. No, m So And I think we wouldn't wanna necessarily do this on ICSI data. I think the ICSI data is too too complex, and there aren't enough real decisions made, you know, that you could clearly identify. I mean there are some, but and not that I've read more than about two meetings. But um I think, w it, you know w well, so one thing that's gonna be interesting is the, I think to me, is the um what can you do because presumably the um D_F_K_I_ folk or somebody is gonna mark up a lot of the data with sort of tags about ontology, right? So we know, you know, somebody will have gone and by hand and sort of marked up som something l like shallow semanticy kinds of things, right, or maybe deeper semanticy things, but that but you know, not really in a in a formal language, but mostly just, you know, tags, right? Um and then the question is, well, how does you know, just the way people typically do this is, you know, let's take these semantic features into or these features into account and see how well we do with the the learning, and then let's throw these features out and see how well we do, um because we know we're gonna have them, because one or two sides is really intr are really interested in this ontology stuff, right? I mean T_N_O_ is and D_F_K_I_ is, right. So there will be some kind of domain semantics. And there will also be presumably some kind of task hierarchy modelling going on in I mean 'cause the whole browser working group um final um sort of presentation on that second day was about, you know, knowing the task, right. And very you know, details about which task is this you know, which sub-task is this little chunk of the meeting addressing and furthering in various ways, right. Which seems like it would be really hard in an open domain. You remember when Steve came in, Steve Whittaker came in at the end and said, you know, they'd spent the morning talking and they came up with this idea. Um No, they said people are talking about tasks, you know. I mean they really they made it clear that they were that that was gonna be heavily No, those are the s those are the scenarios. Those are the user scenarios. The task what what he came back and said that at the end, I mean it actually kind kind of scared me, because it was all based around tasks. And and the domain tasks, okay, they're now they now have to decide how many buttons to put on this thing. I mean it was all about the domain task. And it is really the it and given that, uh it it's really scary to think about what you could possibly do in an open domain where you don't where you don't know what the tasks are. And I don't think you're gonna be able to figure them out uh very reliably. I do, but the whole project is, you know, focussed on AMI hub data as their first priority. N no no no no no, but I mean if the if the if they had th they had this sort of epiphany at the end of this meeting where they decided well why are people browsing this. Well it's all to see how, you know, how the participants were furthering their tasks. Well right. And then, yes, and then there was this confusion about the um last spider phone conference that the messages telling people to call came out after the meeting happened, at least to some of us. So um I mean we'll we'll get that back on track. But I think that um we don't you know, there will be some meeting browser demo that um, you know I I think it's an interesting question of okay well, if you have knowledge, here's what you can do. If you don't have that knowledge you have to do different things and, you know, you can't necessarily Well I think when he does re when he does lear these learning experiments, he should do 'em two ways. Because he will have this opportunity with or without. And then we can see like well without, you know, these tags so you know, Alex was showing um yesterday well without knowing the head word of the utterance at the leaf, you know, you get these kinds of results, and when you add that in, you get these other kinds of results. And you know, I think yeah. Um so I think if we have those features, it's, you know, it's a rare orp or rare opportunity that you can, you know, you can try one head to head against the other. Um and then of course to make sure that you really um Right. Well we won't I mean we won't use 'em till they're ready obviously. So we might do the sort of open domain, I mean a although it will be a closed domain, I mean we we can't use any domain information necessarily. Uh I mean, there probably are a bunch of issues like if you've got a hundred hours of meetings and they all talk about a lot of the same things, then it's ea it it's not really open domain and somehow that sneaks into all the statistics anyway 'cause you don't have such a wide vocabulary and, you know Right. Sure, so we might find that when we move from what we thought was open domain, but on the hub, you know, our results go down when we truly are getting meeting yeah. So I mean that'll be interesting. Okay, well that's good. So okay. So we wanna decide um what Wei-Xun should do next and maybe think about meeting in, I don't know, a month or something. Yeah, mm-hmm. Um so before we do that, can I just ask you a question about chunking? In the beginning you said something about you were doing this chunking um and you're doing it on the ICSI data, but this was did you say it th wasn't picked up in AMI or you I I I was confused about w Mm-hmm. Mm-hmm. And you and you haven't done it yet. Tha that's fine. So that's still on the agenda to do at some point. Yeah. Exactly. That's Well so I mean right. Um or building blocks. I mean so I mean we have to think about how s chunking interacts with the automatic segmentation, you know. Because um Barbara was showing the low level. And I mean it seems to me that chunking should have some kind of so if if your dialogue act segmenter says there's there's a segment here, but your chunker said no, this is a coherent chunk, then y y you know, w I mean maybe they could work together to do a better job than Right. Yeah. Periods. Oh, after uh right. Okay. Right, okay. Good. So it sounds like well w Well how much AMI Well, you wanted to do the chunking on the AMI data, right? Because m So we got stuck at some point on evaluation of chunking, right. The problem was you didn't have ICSI data that was hand labelled for chunks. And we decided labelling the ICSI data for chunks would be probably difficult and maybe not very useful. And so it it mi Okay, if there if there was if there was hand labelled data that you could evaluate again so train on some and evaluate on some, wouldn't that um 'cause you've trained on Switchboard, right, where there there are chunks. So if you had some ICSI data that you could maybe use, I don't know, um as a what's the n there's t there's a training set and then there's the development set and then there's the test set. Maybe you'd use some chunked ICSI stuff as development or something and do some last tweaks and then evaluate. But you know, w h I mean I can see why Wei-Xun didn't wanna spend his time chunking ICSI d Which means we have to get some portion of the data hand-chunked. How much um well how much in Switchboard. Switchboard is all chunked. Right. Right. Well No, you had Penn tree-bank parses, and you made chunks from that. Mm-hmm. Is anybody gonna parse? Is any gonna try to anybody try to parse the AMI data? Well they're using the E_R_G_, right? The which probably has a l maybe finer g uh which I believe has finer grained distinctions than, and different labels, than the Penn tree-bank. It's I'm t it's probably easy to dumb down the H_P_S_G_ parse too. 'Cause the Penn tree-bank parsers are pretty minimal because they didn't wanna commit to a specific grammatical theory. So I mean it probably is worth looking at 'Cause they're doing this with Dan Flickinger, right? Which is the same um parser as the Redwoods Tree-bank was made with that Alex talked about yesterday. So you can look at Redwoods parses um and see how different they are from tree-bank parses. And you might wanna talk to Alex to find out if somebody's already done this turn Redwoods stuff into tree-bank stuff so that they can compare, you know, against results that are reported on tree-bank. Well that was the idea I believe, but um it was supposed to be another tree-bank. Well the good thing about that is that then you know that when you go through and parse you'll be getting the same chunks as when you do this process automatically. 'Cause the problem w when you go through and hand do it is then suddenly you introduce your automatic method and your accuracy goes can go way down. I mean sometimes it doesn't if you know, sort of for part of speech tagging, you know, whether you do it by hand or machine, it's pretty much the same accuracy, right. But They talked about it at the meeting that we went to in Twente. I was thinking Enschede. Why can't I remember the n the f the right name of this place? Yes, Twente. Which You were there Jean. Yeah, okay. No no no no, no no no, I I I know you missed a meeting and s and so there there may have been you missed the one in T_N_O_. On on which project? Sorry, I didn't get that. Verbmobil, right. Yes, yes, they did. It well that's that's what Alex talked about yesterday, right. The parse I mean that's wasn't something she did, but that's the data she's using, right. So we have all that stuff here, you know, the parsed Verbmobil material. So you can look at that and see what the parse trees look like. You don't need to wait for D_F_K_I_. And if it turns out that you think oh, there's no way I can map these into Penn tree-bank style bracketing, then then we have to think of something else, right. But I think they had that in mind. So it's it's worth also talking to her about that. Okay, good. Everybody always says there's no data like more data. You know, I mean the typical thing is to do leave one out cross validation with, you know, eighty nine dialogues. Okay, so I trained on eighty eight and then or I train on one thousand nine hundred and ninety nine and Okay, but we should speak to Steve. We might have some um Well the question the question is what what is chunking gonna help? I mean I think chunking could help automatic dialogue act segmentation. I think chunking could help, as you mentioned, named entity recognition. Um Mm-hmm. Mm-hmm. Well we'd wanna make sure we're pretty happy that it's reliable because Mm yeah. I mean otherwise you get these very bizarre results because the data is not really very good. So um but I and nobody has expressed like massive interest in t in chunking. So I think, you know, nobody's uh waiting for it. But but we could we c I think we should talk to Barbara about how to do some joint experiments or or if she's gonna be sending other people, maybe Liz or whoever comes over next time. Uh and it would actually be great if it was Liz, because I think she is the one who did the dialogue act segmentation automatic She just said that hopefully by the next meeting she would be bringing someone else, or someone else would be coming instead of her, remember? The th not to have not to have it done, but just to talk about how it could interact with automatic dialogue act segmentation. Um and we I guess we should talk to Yoshi about about it in relationship to named entities. Who's uh who's doing named entities besides Yoshi? Okay. So we could talk to Vasilis about that as well. And then So one thing that would be very helpful for me because I haven't read I think we're in a situation where Jean, you know, has read a lot of literature and I've read a lot of literature and we there is some overlap, but there's also quite a big space where we don't know the literature from the other side. And since you're gonna be reading it, I mean it would be probably be useful if you made some kind of, you know, report that described, you know, okay, this what these people you know, this is, you know, a synthesis okay, here's the basic idea behind exchange structure and, you know, here's a couple of you know, here's the two most relevant works that I've read or something like that, you know, to just Or even just like a paragraph of, you know, the basic idea. Just because, you know, obviously I'm not gonna go rea off and read all this kind of stuff because I don't have time. And you probably don't have time to read all this collaborative prol problem solving. So Well in any way, every time you write a paper, you've got to go back and do this related work thing anyway. So it just it just makes it a lot easier um to get that common ground I think, you know. I think it's a real advantage that we've read different things and come at it from a different side because I think this ha they the the previous approaches haven't really, you know, taken you know, the good features from, you know, both sides and and tried to put them together. Yeah, me too. I can send you a l Well I think we both wanna send w well, I don't know, maybe you've already done this, but I you know, I will send you or we can, you know, chat off-line for a minute a about a list of things that I think you really need to take a look at. And Jean, probably maybe you should do the same. Um And if Sabrina has any ideas, we'd be happy to hear from Sabrina as well. Um Mm-hmm. Okay. Yeah. Mm-hmm. Mm-hmm. M Mm-hmm. Hmm. Yeah. Mm-hmm. Well I think we need to um, like you mentioned earlier, um factor in what this might be used for in the end. And so even if it we won't it won't be used, maybe if it would need some higher level analysis before it could be directly used for that purpose, we should still keep that in mind. And so I think going through the presentations that um were given at the work package six meeting where um and I think it was um it wasn't wha was it Anita or was it oh I've forgotten her name, the other woman um from Twente, you know, they talked about they did they had done a lot of questionnaires, and they got some user requirements out of that. And then Steve presented um the results of following people around and looking at how they used their notes from meetings. And and so that would give us some ideas about, you know, what people want, you know, out of meeting records. But then we should also add things that we think are interesting. But just to have to keep some of these goals in mind 'cause they're t o one scheme might let you answer this set of questions but wouldn't help you at all with another set of questions. So should we expect that all the slides from like the work package six meeting or the joint work package five work pa will be u up on one of the work package web sites? And if not, how do we make that happen? Hmm okay. Some of this was presented in at least the one spider phone or the second spider phone telecon I went to for work package six. So if they're not on the web we can try to look at that, you know, at the materials that were presented in that telecon without necessarily listening to the whole telecon. Yeah? Wh And Steve m and his I've forgotten the name, is it was it Simon, his yeah. They they've I think they may have written this up um in a report um, but I wou would hope that's on the work package six web site as well. I do as i once he's read a couple more I mean he's just completely missed out the collaborative problem solving world, which I d I really think we need to take into account. I mean you may take it into account and say no, you're wrong, and here's why, but I I I'm gonna need some convincing. Or t or or two and see this one gives you this and this g and can't you know, are they fundamentally incompatible. Or can we yeah, mm-hmm. Okay. I think this is gonna take about three weeks 'cause he's got plenty of reading to do. So maybe we get together again in the yeah, I mean, I'll send them the references immediately. Is that enough time for you do you think? Is that enough time for you to to read and think about If y if if it's too tight, you know, I think m we can say four. Or are you gonna be on holiday? Yeah. Maybe four weeks? Okay. Okay. So we should get together probably in about four weeks, yes. So when are you gonna bring this beautiful baby in to see everybody? Good, good. That's great. Yep. Well good luck solving your problem. I have to tick and tick? It does something weird when you do that.
